Neural networks can be used to analyze images for a variety of purposes. For example, some neural networks can examine images to identify objects depicted in the images. Some sets of images are or can represent frames of a video showing relative movement between an object depicted in the images and a viewer of the images. Optical flow describes this relative motion of the objects with respect to the observer in a visual scene. Accurate and effective calculation of optical flow can be used in various computer vision applications, such as object detection, tracking, movement detection, robot navigation, three-dimensional (3D) reconstruction and segmentation, etc.
But, accurate and effective calculation of optical flow can be difficult to achieve. This calculation amounts to finding pixel correspondences between consecutive frames, which is a time-consuming task. At times, there is no exact pixel intensity correspondence, or the solution is not unique, which frustrates or prevents determination of the optical flow.